Clinical Study of Artificial Intelligence in Imaging Diagnosis of False Positive Lesions of Pulmonary Nodules

By He Sun^1,2, Jiaheng Wei¹, Junfu Wang², Zhanyue Pang^2,3, Liangming Zhu^3,4

Affiliations

1. School of Clinical Medicine, Weifang Medical University, Weifang, Shandong, China
2. Department of Thoracic Surgery, Jinan Central Hospital, Jinan, Shandong, China
3. Department of Thoracic Surgery, Jinan Central Hospital, Shandong University, Jinan, Shandong, China
4. Department of Thoracic Surgery, School of Medicine, Cheeloo College of Medicine, Shandong University, Jinan, Shandong, China

doi: 10.29271/jcpsp.2023.10.1087

ABSTRACT
Objective: To determine the accuracy of diagnosis of pulmonary nodules using artificial intelligence method.
Study Design: Observational study.
Place and Duration of the Study: Department of Thoracic Surgery, Jinan Central Hospital, Jinan, China, from January 2020 to May 2021.
Methodology: An analysis of clinical characteristics exhibited by 32 patients initially diagnosed with malignant tumours through imaging (LDCT) and artificial intelligence (AI), was reclassified as having benign lesions following surgical intervention. Quantitative parameters were assessed, including CT mean value, kurtosis, skewness, solid ratio, and the ratio of length to short diameter, within a cohort of 32 benign patients juxtaposed with 58 patients diagnosed with lung cancer during the same time frame. The AI-derived parameters were subjected to Mann-Whitney U non-parametric test.
Results: A total of 32 benign pulmonary lesions were evaluated that were initially misdiagnosed as malignant prior to surgery. These lesions displayed an average length of (18.56 ± 12.16) mm, with the majority characterised as solid (68.8%). Notably, a substantial proportion of these lesions exhibited imaging features akin to malignant growths. The AI-derived quantitative parameters of the 32 benign cases and the 58 malignant cases revealed statistical significance in average CT value and solid ratio. However, statistical significance was not established for kurtosis, skewness, or the ratio of length to short diameter. The area under the Receiver Operating Characteristic (ROC) curve for average CT value and solid ratio stood at 0.71 and 0.705, respectively.
Conclusion: Among the cases initially misdiagnosed as malignant yet subsequently identified as benign, a notable number of these instances were solid nodules, often resembling malignant lesions in imaging characteristics. There was moderate discriminatory capacity for average CT value and solid ratio, rendering them valuable tools for distinguishing between benign and malignant lesions within this particular cohort. This underscores their high diagnostic significance.

Key Words: Artificial intelligence, Benign lesions of lung, Lung cancer, Quantitative parameters, Postoperative.

INTRODUCTION

Lung cancer holds the highest mortality rate among malignant tumours globally, as supported by the statistical data.¹ In China, the incidence and fatality rate of lung cancer are also prominently elevated among malignancies.² In the realm of medical advancements, artificial intelligence (AI) has rapidly emerged as a transformative technology, finding widespread utility.³ Researchers, both domestic and international, have harnessed AI for analysis and exploration of pulmonary nodules.⁴ Of these technologies, Convolutional Neural Networks (CNNs), as distinguished deep learning algorithms, have garnered substantial attention for their role in diagnosing pulmonary nodules.⁵

Notably, studies reveal that 2D and 3D CNNs boast a detection sensitivity of approximately 95% for pulmonary nodules.^6,7 Computed tomography (CT) assumes a pivotal role in the early detection of lung cancer through nodule screening, thereby significantly mitigating the mortality rate.⁸ According to the relevant literature,^9,10 the current detection rate of chest CT for pulmonary nodules has reached 21-33%. In the developing countries’ areas with high incidence of tuberculosis, chest CT can detect more pulmonary lesions.¹¹

The contemporary proliferation and rapid advancements in low dose computed tomography (LDCT) have substantially bolstered pulmonary nodule detection rates. LDCT exhibits heightened sensitivity in the early lung cancer diagnosis; however, it is characterised by a notable false-positive rate, diminished specificity, and susceptibility to physicians' subjectivity. AI, in its capacity to rapidly differentiate between benign and malignant pulmonary lesions across an extensive array of CT images, emerges as a crucial tool.^12,13 The management of benign nodules typically involves vigilant observation. For lesions defined as intermediate in nature, bronchoscopy and lung biopsy are conventionally advised. Nevertheless, certain nodules remain diagnostically elusive due to factors such as location and size, compelling patients to seek resolution through video-assisted thoracoscopy, driven by their comprehension of the tumorous condition and ensuing psychological stress. Notably, patients confronted with indeterminate pulmonary lesions often opt for surgical intervention to ascertain lesion nature, thereby imposing a multifaceted toll on their well-being, mental equilibrium, and society at large. Addressing the imperative of minimising the frequency of surgical interventions for benign cases assumes critical significance.

In this study, the aim was to determine the average CT value and solid ratio for improving the accuracy of diagnosis of pulmonary nodules based on artificial intelligence method, which has contributed to develop a novel tool to classify pulmonary nodules.

METHODOLOGY

A total of 244 patients including 35 malignant tumours and 209 lung cancer were collected from Jinan Central Hospital in this study. The exclusion criteria was patients who received pulmonary lesion resection from January 1, 2020 to May 31, 2021, the thickness of chest CT layer (1mm or 1.25mm) and the postoperative pathological conditions such as preoperative bronchoscopy, lung puncture biopsy, lymph node biopsy, the history of other tumours and chemoradiotherapy, and findings of squamous cell carcinoma or small cell lung cancer. Finally, 32 benign cases and 58 malignant cases were finally included in this study.

The CT equipment models, GE750HD (USA), Siemens SOMATIOM Definition AS (Germany) were used for the study.

The CT scanning parameters were tube rotation time 0.5s, standard soft tissue reconstruction algorithm, slice thickness 1mm or 1.25mm, reconstruction interval 1mm or 1.25mm × DFOV 200mm-410mm; tube voltage 120V, tube current 160mA, lung window: 800HU / 750HU; and mediastinal window: 400HU / 40HU. Utilising the capabilities of the Infervision artificial intelligence assisted diagnosis software, hinged on deep neural network technology for auxiliary analysis of medical images, the quantitative parameters of patient CT images underwent analysis across the stages of anomaly detection (lesion identification), differential diagnosis (intelligent localization), segmentation measurement (quantitative analysis), image registration (intelligent tracking), and generation of intelligent reports.

The dataset was statistically processed and visualised using SPSS version 27 software. Expressing measurement data as mean ± standard deviation (mean ± SD), while adoption rates of count data (%) were employed. The normality of data distribution was assessed using the S-W test. Non-normally distributed continuous variable data were represented using the median and interquartile range M (Q25, Q75). The comparison between the two groups was executed through the Mann-Whitney U test. Parameters exhibiting statistical significance underwent further analysis via ROC curve calculations. A statistical significance threshold of p <0.05 was adopted.

RESULTS

This investigation encompassed 32 patients who were initially diagnosed with malignant lesions preoperatively, only to be reclassified as having benign lesions postoperatively. Among the participants, the male-to-female ratio stood at 18:14, with an average age of 58.87 ± 8.44 years. The average long diameter measured 18.56 ± 12.16 mm, with lesions ≥ 8mm constituting 93.7% of cases. Notably, 22 cases (68.8%) presented as solid lesions. The observations revealed the presence of lobulation signs in 21 cases (65.6%), spiculation signs in 17 cases (53.1%), pleural traction signs in 10 cases (31.3%), vascular convergence signs in 3 cases (9.4%), and vacuole signs in 1 case (3.1%). Among them, 20 patients (62.5%) were found to have pulmonary lesions due to health examination. All the 32 patients with benign lesions underwent intraoperative rapid pathological examination, of which 1 case was diagnosed as atypical hyperplasia during operation and pulmonary inflammatory lesions were indicated by routine pathological report after operation. The coincidence rate between rapid frozen pathological section and routine pathological section was 96.9%. Postoperative routine pathological types showed that the top three were inflammatory lesions in 13 cases (40.6%), atypical hyperplasia in 11 cases (34.4%) and hamartoma in 6 cases (18.8%, Table I).

Table  I:  Clinical  and  imaging  features  of  lung  benign  lesions.

Variable	Data
Age (year)	58.87±8.44
Gender (male)	18(56.3%)
Total number of evaluable nodules	32
Location       Left upper lobe       Left lower lobe       Right upper lobe       Right middle       Right lower lobe       Peripheral type       Central type	- 5(15.6%) 7(21.9%) 9(28.1%) 6(18.8%) 5(15.6%) 31(96.9%) 1(3.1%)
Diameter of pulmonary lesions (mm)       <8       8-20       20-30       >30	- 2(6.3%) 22(68.8%) 3(9.4%) 5(15.6%)
Lung signs       Spicule sign       Vacuole sign       Lobulation sign       Vessel convergence sign       Pleural traction sign       No special signs	- 17(53.1%) 1(3.1%) 21(65.6%) 3(9.4%) 10(31.3%) 11(34.4%)
Density classification       Pure ground glass nodule       Partial solid nodule       Solid nodule	- 3(9.4%) 7(21.9%) 22(68.8%)
Pathological results       Inflammatory lesions       Atypical hyperplasia       Hamartoma       Other	- 13(40.6%) 11(34.4%) 6(18.8%) 2(6.3%)
Detected by physical examination       Yes       No	- 20(62.5%) 12(37.5%)

Table II: Analysis of AI quantitative parameters by statistical method.

Quantitative parameters median (IQR)	Benign lesion (n=32)	Malignant lesion (n=58)	p-valuea	AUC (p-value)	Asymptomatic 95% Confidence Interval	ppv	npv
					Lower-Upper
Average CT value	-95 (-360.75, 33)	-458 (-572, -66.75)	0.001a	0.71 (0.001)	0.6-0.819	87.4%	76.9%
Kurtosis	-0.92 (-1.16, -0.37)	-0.8 (-1.11, -0.11)	0.451a	0.452 (0.451)	0.324-0.580	32.5%	26.3%
Skewness	0.1 (-0.02, 0.59)	0.35 (0.06, 0.67)	0.052a	0.376 (0.052)	0.246-0.505	30.6%	25.7%
Solid ratio	76.56 (15.46, 97.38)	12.08 (1.15, 77.86)	0.001a	0.705 (0.001)	0.598-0.812	84.6%	73.5%
Ratio of long diameter to short diameter	1.28 (1.17, 1.4)	1.33 (1.17, 1.51)	0.466 a	0.453 (0.466)	0.331-0.576	33.5%	29.7%
Mann-Whitney U-test; the difference was statistically significant (p <0.05).

Figure 1: CT imaging of benign lesions diagnosed as malignant before operation and malignant lesions confirmed by operation and pathology. (A, B) The nodule of the upper lobe of the right lung with lobulation, burr and pleural traction. Surgical pathology: organised pneumonia. (C, D) Left upper lobe nodule with lobulation and burr. Surgical pathology: atypical hyperplasia. (E, F) Nodule of upper lobe of left lung, superficial lobulation. Surgical pathology: hamartoma. (G, H) Nodule of upper lobe of right lung. Surgical pathology: adenocarcinoma in situ. (I, J) The nodule of the lower lobe of the right lung with lobulation and vascular penetration. Surgical pathology: adenocarcinoma in situ with microinvasion. (K, L) The lower lobe of the left lung with lobulation, burr and pleural traction, bronchial amputation. Surgical pathology: invasive adenocarcinoma.

Infervision artificial intelligence software was employed to determine AI-derived quantitative parameters (CT mean value, kurtosis, skewness, solid ratio, ratio of length to short diameter) across 32 patients with benign lesions and 58 patients with malignant lesions. The outcome of the Mann-Whitney U test demonstrated that in comparison to the malignant lesion group, the benign lesion group exhibited higher average CT values and a greater proportion of solid lesions, both achieving statistical significance (p <0.05). Conversely, statistical significance was not observed for kurtosis, skewness, and the ratio of length to diameter (p >0.05). Detailed results can be found in Table II.

Figure 2: Result of Infervision artificial intelligence software for benign lesions.

Figure 3: Result of Infervision artificial intelligence software for malignant lesions.

Representative CT images of benign and malignant lesion cases are illustrated in Figure 1, while artificial intelligence software-aided analyses of benign and malignant lesions are depicted in Figures 2 and 3, respectively.

As shown in Table II, ROC analysis showed AUC of average CT value and solid ratio was 0.71 (p = 0.001), and 0.705 (p = 0.001), respectively. The best critical values of CT average and solid ratio were -502.5 HU (sensitivity = 90.6%, specificity = 46.6%) and 7.96% (sensitivity = 93.8%, specificity = 44.8%).
 

DISCUSSION

The current study delved into an examination of the clinical and imaging characteristics of 32 patients initially diagnosed with malignancy, only to be reclassified postoperatively as having benign lesions during the period spanning from January 2020 to May 2021 within the study hospital. Most patients were identified through the routine health examinations, underscoring a heightened awareness of health among individuals. It is imperative to improve the precision of identifying potentially malignant benign lesions, thereby alleviating unwarranted patient burdens. As per prevailing guidelines,¹⁴ where malignancy risk ranges from 3 to 68% for uncertain nodules, pursuing CT-guided percutaneous lung biopsy or bronchoscopic biopsy to obtain definitive diagnoses represents a feasible approach, effectively reducing the prevalence of unnecessary surgical interventions.

The evaluation encompassed a total of 32 pathologically established benign lesions, with a noteworthy 93.7% exhibiting sizes exceeding 8cm. Among these, 9.4% were pure ground glass nodules, while 90.7% presented as mixed ground glass and solid lesions. Simultaneously, malignancy-associated imaging attributes like the spiculation sign, vacuole sign, lobulation sign, vascular convergence sign, and pleural traction sign were evident. Importantly, a substantial proportion of suspected malignant benign lesions manifested imaging features resembling malignancies. The existing literature has extensively explored the attributes of pulmonary nodules.¹⁵ The authors can augment assessments by scrutinising characteristics such as calcification and fat density indicative of benign lesions, thereby enhancing the evaluation of suspicious lesions. Additionally, this study found a notable 96.9% agreement between rapid frozen pathological assessments and conventional pathology reports. This underscores the significance of rapid frozen pathology in expediting operations and refining the scope of necessary lung resections, affirming its indispensable role. Furthermore, this research revealed that the majority of benign lesions treated within our hospital were inflammatory in nature, followed by hamartoma and atypical hyperplasia. Clinicians routinely implement empirical anti-inflammatory treatments for suspicious lesions. However, some patients may receive non-standardised drug regimens, and specific infectious lesions might not resolve promptly, leading to diagnostic ambiguity. To address this, consistent monitoring and standardised antibiotic usage are advocated. Con-currently, targeted laboratory tests for specific infections can be instrumental in enhancing the accuracy of assess-ments for suspicious lesions.

Prior research has indicated that the average and maximum diameters observed through CT scans act as independent risk factors of notable value in discerning between benign and malignant pulmonary lesions.¹⁶ Concurrently, density attributes such as kurtosis and skewness of pulmonary nodules have also displayed discriminatory potential between the two categories,¹⁷ albeit often in conjunction with other parameters.¹⁸ A significant contrast in solid components had been observed between malignant and benign pulmonary nodules.¹⁹ Building upon these findings, this study employed Infervision artificial intelligence software to determine AI-derived quantitative parameters (average CT value, kurtosis, skewness, solid ratio, ratio of length to short diameter). A comparison was drawn between 32 patients with benign lesions and 58 patients with pathologically confirmed malignant lesions. Remarkably, the average CT value and solid ratio emerged as the two parameters exhibiting statistical significance. In imaging features, because there are different degrees of morphological similarity between benign and malignant lesions, kurtosis, skewness and the ratio of length to short diameter could not provide a reference for the differential diagnosis of malignant benign lesions. For suspected malignant benign lesions, the average value and solid ratio of CT could still provide some suggestions for the differentiation of benign and malignant lesions. The average value of CT represented the mean value of CT value of pulmonary nodules, and the solid proportion represented the proportion of solid components in pulmonary nodules. These two indicators represented the density characteristics of pulmonary nodules. Through the analysis of ROC curve, the best critical values of CT average and solid ratio were -502.5 HU (sensitivity = 90.6%, specificity = 46.6%) and 7.96% (sensitivity = 93.8%, specificity = 44.8%). Because the sensitivity of these two kinds of parameters is more than 90%, it can be found that the rate of missed diagnosis is low. When the average value of CT is more than -502.5 HU, it is more inclined to suspected malignant benign lesions, when the average value of CT is less than -502.5 HU, it is more inclined to malignant lesions; when the solid proportion is more than 7.96%, it is more inclined to suspected malignant benign lesions, and when the average CT is less than 7.96%, it is more inclined to malignant lesions. Most of these benign lesions are inflammation and hamartoma with more solid components. The higher the solid component, the higher the mean CT value. The related literature²⁰ suggests that most of the benign nodules of the lung are solid nodules and most of the malignant nodules are partial solid nodules. In this study, it was found that the average CT and solid ratio of these benign nodules were higher than those of malignant nodules, which was consistent with the conclusion of the proportion of solid components in benign and malignant nodules. The average area of CT in ROC curve is 0.71and the area of solid ratio is 0.705, indicating that the average value of CT and solid ratio are of high predictive value, which can be used to distinguish benign lesions suspected of malignancy from malignant lesions.

The limitations of this study stemmed from the fact that it was a retrospective case analysis. Being a single-centre study, the sample size and index parameters are less. The authors hope to increase more samples in the future, take multi-centric, prospective research, at the same time, with the development of AI intelligent software, as that will lead to more indicators for follow-up researches. Despite the aforementioned limitations, the authors are confident that this study will help to better identify benign lesions suspected of malignancy and increase people's understanding of lesions that are difficult to define. It will be helpful to the practice of clinicians and radiologists.

CONCLUSION

The presence of benign nodules displaying imaging attributes akin to malignant lesions, presents diagnostic challenges. Upon juxtaposing postoperative pathological findings of benign and malignant groups, the ROC curve analysis of average CT value and proportion of solid lesions demonstrated a moderate area under the curve, signifying their substantial diagnostic significance. These outcomes hold the potential to guide efforts aimed at mitigating the proportion of surgical interventions in cases of benign pulmonary lesions.

ETHICAL APPROVAL:
The study was approved by the Ethics Committee of Jinan Central Hospital, Shandong, China (No. 2022-244-01, dated 2022.10.07).

PATIENTS’ CONSENT:
Informed consents were obtained from the patients for performing the tests and to publish the obtained data.

COMPETING INTEREST:
The authors declared no competing interest.

AUTHORS’ CONTRIBUTION:
HS: Design, acquisition and analysis of data, and writing of the manuscript.
JW, JW, ZP: Interpretation and discussion of results.
LZ: Proofreading and final approval of the final manuscript.
All authors approved the final version of the manuscript to be published.

REFERENCES

1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2020. CA Cancer J Clin 2020; 70(1):7-30. doi: 10.3322/caac.21590.
2. Zheng RS, Sun KX, Zhang SW, Zeng HM, Zou XN, Chen R, et al. Zhonghua Zhong Liu Za Zhi 2019; 41(1):19-28. doi: 10.3760/cma.j.issn.0253-3766.2019.01.005.
3. Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer 2018; 18(8):500-10. doi: 10.1038/s41568-018-0016-5.
4. Ather S, Kadir T, Gleeson F. Artificial intelligence and radiomics in pulmonary nodule management: current status and future applications. Clin Radiol 2020; 75(1):13-9. doi: 10.1016/j.crad.2019.04.017.
5. Lee SM, Seo JB, Yun J, Cho YH, Vogel-Claussen J, Schiebler ML, et al. Deep learning applications in chest radiography and computed tomography: Current state of the art. J Thorac Imag 2019; 34(2):75-85. doi: 10.1097/RTI.00000 00000000387.
6. Tran GS, Nghiem TP, Nguyen VT, Luong CM, Burie JC. Improving accuracy of lung nodule classification using deep learning with focal loss. J Healthc Eng 2019; 2019:5156416. doi: 10.1155/2019/5156416.
7. Gruetzemacher R, Gupta A, Paradice D. 3D deep learning for detecting pulmonary nodules in CT scans. J Am Med Inform Assoc 2018; 25(10):1301-10. doi: 10.1093/jamia/ ocy098.
8. Zhao L, Bai CX, Zhu Y. Diagnostic value of artificial intelligence in early-stage lung cancer. Chin Med J (Engl) 2020; 133(4):503-4. doi: 10.1097/CM9.0000000000000634.
9. Croswell JM, Baker SG, Marcus PM, Clapp JD, Kramer BS. Cumulative incidence of false-positive test results in lung cancer screening. Ann Intern Med 2010; 152(8):505-W180. doi: 10.7326/0003-4819-152-8-201004200-00007.
10. Gould MK, Tang T, Amy Liu IL, Lee J, Zheng C, Danforth KN, et al. Recent trends in the identification of incidental pulmonary nodules. Am J Respir Crit Care Med 2015; 192(10):1208-14. doi: 10.1164/rccm.201505-0990OC.
11. Bai C, Choi CM, Chu CM, Anantham D, Chung-Man Ho J, Khan AZ, et al. Evaluation of pulmonary nodules: Clinical practice consensus guidelines for asia. Chest 2016; 150(4):877-893. doi: 10.1016/j.chest.2016.02.650.
12. Gursoy Çoruh A, Yenigun B, Uzun Ç, Kahya Y, Buyukceran EU, Elhan A, et al. A comparison of the fusion model of deep learning neural networks with human observation for lung nodule detection and classification. Br J Radiol 2021; 94 (1123):20210222. doi: 10.1259/bjr.20210222.
13. Du W, He B, Luo X, Chen M. Diagnostic value of artificial intelligence based on CT image in benign and malignant pulmonary nodules. J Oncol 2022; 2022:5818423. doi: 10. 1155/2022/5818423.
14. Gould MK, Donington J, Lynch WR, Mazzone PJ, Midthun DE, Naidich DP, et al. Evaluation of individuals with pulmonary nodules: When is it lung cancer? Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest 2013; 143(5 Suppl): e93S-e120S. doi: 10.1378/chest.12-2351.
15. Bartholmai BJ, Koo CW, Johnson GB, White DB, Raghunath SM, Rajagopalan S, et al. Pulmonary nodule charac-terization, including computer analysis and quantitative features. J Thorac Imag 2015; 30(2):139-56. doi: 10. 1097/RTI.0000000000000137.
16. Xinli W, Xiaoshuang S, Chengxin Y, Qiang Z. CT-assisted improvements in the accuracy of the intraoperative frozen section examination of ground-glass density nodules. Comput Math Methods Med 2022; 2022:8967643. doi: 10. 1155/2022/8967643.
     
17. Kamiya A, Murayama S, Kamiya H, Yamashiro T, Oshiro Y, Tanaka N. Kurtosis and skewness assessments of solid lung nodule density histograms: Differentiating malignant from benign nodules on CT. Jpn J Radiol 2014; 32(1):14-21. doi: 10.1007/s11604-013-0264-y.
18. Borguezan BM, Lopes AJ, Saito EH, Higa C, Silva AC, Nunes RA. Solid indeterminate nodules with a radiological stability suggesting benignity: A texture analysis of computed tomography images based on the kurtosis and skewness of the nodule volume density histogram. Pulm Med 2019; 2019:4071762. doi: 10.1155/2019/4071762.
19. Park CM, Goo JM, Kim TJ, Lee HJ, Lee KW, Lee CH, et al. Pulmonary nodular ground-glass opacities in patients with extrapulmonary cancers: what is their clinical significance and how can we determine whether they are malignant or benign lesions? Chest 2008; 133(6):1402-9. doi: 10.1378/ chest.07-2568.
20. Wan YL, Wu PW, Huang PC, Tsay PK, Pan KT, Ngoc Trang N, et al. The use of artificial intelligence in the differentiation of malignant and benign lung nodules on computed tomograms proven by surgical pathology. Cancers (Basel) 2020; 12(8):2211. doi: 10.3390/cancers12082211.